Small Change, Big Impact: Optimizing GPU Memory Usage

This pull request introduces a small yet impactful optimization to the GPU memory usage in the AutoModel class, leveraging PyTorch's Automatic Mixed Precision (AMP) feature. By simply wrapping the model's inference code within the autocast context manager from torch.cuda.amp, we significantly reduce memory usage during GPU operations. This small change is particularly beneficial for users with lower-memory GPUs, as it allows more efficient use of available resources.

Key Change

• Integrated with autocast(): within the translate_sentences method of AutoModel.
• This minor change enables mixed-precision computation, reducing the GPU memory footprint by using float16 precision where possible without affecting model performance.

Impact

• Remarkable Efficiency Gains: This change drastically reduces GPU memory consumption. It's a game-changer for users with GPUs that have lower memory, allowing for smoother operation and reducing out-of-memory errors.
• Maintains Compatibility: The update maintains full backward compatibility. On systems without a GPU or those not supporting AMP, autocast simply becomes a no-operation, preserving existing functionality.
• Expands Accessibility: By making the package more memory-efficient, we're opening the doors to a broader range of users who might have been limited by hardware constraints.